Detecting Relationships Through Large-scale Photo Analysis

The popularity of online social networks has changed the way in which we share personal thoughts, political views, and pictures. Pictures have a particularly important role in the privacy of users, as they can convey substantial information (e.g., a person was attending an event, or has met with another person). Moreover, because of the nature of social networks, it has become increasingly difficult to control who has access to which content. Therefore, when a substantial amount of pictures are accessible to one party, there is a very serious potential for violations of the privacy of users. In this paper, we demonstrate a novel technique that, given a large corpus of pictures shared on a social network, automatically determines who is dating whom, with reasonable precision. More specifically, our approach combines facial recognition, spatial analysis, and machine learning techniques to determine pairs that are dating. To the best of our knowledge, this is the first privacy attack of this kind performed on social networks. We implemented our approach in a tool, called Creepic, and evaluated it on two real-world datasets. The results show that it is possible to automatically extract non-obvious, and nondisclosed, relationships between people represented in a group of pictures, even when the people involved are not directly part of a connected social clique. DOI 10.1515/popets-2015-0004 Received 11/22/2014; revised 2/16/2015; accepted 2/16/2015.